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Teaching Psychometrics in South Korea Through A 
Reunification Attitude Scale Class Project 
Sandra K. Webster 
Fulbright Lecturer Korea University 
Professor of Psychology, Westminster College 

Abstract 

The introduction of a team term project into a Korean psychometrics class is 
described. Students developed an item pool of attitude statements regarding the 
reunification of South Korea and North Korea. Then teams of students used the item 
pool to develop attitude questionnaires, survey other students, analyze the results and 
recommend which items to include in an Attitudes Toward Reunification Questionnaire. 
Evaluation of the project focuses on practical course administration issues and cross- 
cultural considerations. 



Objectives 

The objectives of this teaching activity were to give students experience in the 
use of scientific problem solving, to help them understand the reasons behind the 
principles of scale construction and to give them an appreciation of the methods of 
psychometrics as they used them in scale construction. 

Course Context 

The psychometrics course is an upper level psychology major course with no 
prerequisites. It was unusual in the Fall of 2000 because it was taught in English by an 
American instructor. It was also unusual because it included the student team term 
project. The typical Korean undergraduate course does not include student centered 
active learning, but relies on a traditional lecture format. The students reported that the 
project was their first experience of using course content to develop and complete an 
empirical project. 

During a psychometrics course students should learn about test development, 
evaluation and application. Course content pertaining to test construction, reliability and 
validity can be both difficult and tedious at the same time. A deeper understanding and 
appreciation of that content can be achieved if students actually apply knowledge of 
psychometrics to develop a psychological measurement instrument. The particular 
psychological characteristic (variable) to be assessed should be one that is of clear 
relevance to the students and of interest in current psychological research. Attitudes 
toward Korean reunification fit these criteria. 

The class project began shortly after the announcement of Kim Dae Jung’s Nobel 
Peace Prize. Although many Koreans were very proud of their President’s efforts toward 
reunification there seemed to be a broad range of attitudes toward it. Understanding 
these attitudes can contribute to studies of peace and conflict resolution. Korea claims 
a 5000 year history with the unified Shilla Kingdom marking the beginning of a 
combined Korea dated at the year 668. The entire Korean Peninsula was one country 
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for the next 12 1/2 centuries. The Korean War resulted in its division into North Korea 
and South Korea in 1 953. Almost half a century later North Korea is recognized as one 
of the world’s last remaining totalitarian communist dictatorships. In contrast, South 
Korea is a very dynamic, highly developed, capitalist democracy. Both countries are led 
by Kims. President Kim II Jong is the first communist dictator in the history of the world 
to succeed his father as the leader of his country. President Kim Dae Jong received the 
Nobel Peace Prize for his attempts to broker peace and eventual reunification with 
North Korea. 

Early in the semester the class project had been defined as a Korean version of the 
Belief in Equality Scale (Gray, 1996 ). This topic produced cooperation from the 
students but no enthusiasm. They explained that Korea’s Confucian heritage made 
equality irrelevant to them. Reunification of the two Koreas on the other hand was their 
issue. Students in South Korea are very politically active. Demonstrations are the norm. 
A previous South Korean dictator has been forced from power after nationwide 
demonstrations that started with Korea University students. Their perceived political 
power and the timeliness of the topic combined to increase student motivation for the 
course project. 



Project Stages 

The class project was divided into four stages: a) development of an item pool, 
b) preliminary draft instrument review, c) data collection and scale construction and d) 
report. 

Item Pool Development . Each class member wrote ten attitude statements regarding 
reunification of the two Koreas (five against reunification and five in favor of 
reunification). They were instructed to write statements that reflected a full range of 
attitudes. Although the students were asked to submit the items in both Korean and 
English the majority of students submitted initial items only in English. 

Draft Instrument Review . In the second stage of the task the class members were 
assigned to one of three teams (5-7 members per team). Each team leader (volunteer) 
was given instructions for stage 2 of the task. This stage involved selecting a method of 
scale construction (Guttman, Thurstone or Likert) and using that method with the class 
generated attitude statements to construct a reunification attitude scale. During this 
stage team members identified the information needs required to perform the task. 
Questions they asked included: 

• What should they do about redundant or ambiguous items? 

• How many items should be in the final scale? 

• What is the appropriate number of judges or raters for the preliminary items? 

• What specific formula should they use for estimating reliability? 

• How does the theory of test construction relate to the measurement of attitudes 
toward reunification? 



This stage was particularly challenging to the students because, as they reported, they 
had not previously participated in the active application of psychology content for an 
empirical project. When the objectives were again explained to them they seemed to 
enthusiastically rise to the challenge. In their enthusiasm two of the three teams began 
data collection prematurely (before instructor review). 

Data Collection and Analysis . In the third stage the students collected data, performed 
item analyses, determined the items for their final scales and calculated scale reliability. 
Shared workloads within the team made the project feasible in the constricted time 
available within a semester. Each team member was responsible for administering a 
set of the surveys. Although all the students were either juniors or seniors their 
previous course work in psychology ranged from none to as many as 12 courses. Each 
team had students who had previously studied statistics and those who had not. They 
were encouraged to share the work fairly, with those having more statistics knowledge 
leading in that part of the project while the others could lead in other parts. 

Report . Team reports were in three parts. Each team completed a written Method 
section that described the sample of raters or judges used to develop the scale, the 
materials used (including their preliminary instrument coded with original item pool 
numbers) and the procedure used to collect the ratings or judgments. An oral team 
report was given that included: a) the objective of the attitude assessment, b) an 
explanation of why the particular method of scale construction was chosen, c) the final 
items selected for the scale with an explanation of the criteria used by the team for item 
inclusion. The oral reports closed with the scale’s internal consistency measures and 
the team’s suggestions for the next steps to determine the reliability and validity of their 
scale. The third component of the report included the raw data, an electronic form of 
the data and all statistical analyses that were used to construct the scale. 

Evaluation and Recommendations 

The benefits of this class exercise were that students increased their motivation 
to understand the concepts of psychometrics. Through using a team approach they 
were able to work together to discover what essential features of scale construction they 
needed to review or study more deeply. They were also able to understand first hand 
the problem solving process of psychological measurement. One question on the final 
exam asked students to describe the most important thing that they had learned in the 
course and to explain why it was most important. A majority of the 18 students chose to 
talk about the depth of learning that they had achieved through applying the techniques 
in the project as opposed to simply reading about them. Of those students who did not 
graduate (January graduation), 2/3 elected to take an advanced research methods 
course the next semester that would also include active team based projects. Because 
the number of students in the second class who had also completed the first class was 
so high and student interest in reunification was still strong that class used the attitude 
scales developed in the psychometrics class for the basis of an investigation of the 
effects of temporal focus on Korean University Student attitudes toward reunification. 
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From a course administration perspective dividing the project into several clear 
parts with specific deadlines and quick faculty review of those parts was very important. 
Each part should be assigned as students have covered the course content to be 
applied. Too much information, given too soon would confuse the students. It could 
also serve to discourage them with the magnitude of the project. Some students were 
frustrated with the ambiguity of course requirements. However, it is important for 
students to learn about the iterative nature of research and the value of flexibility. The 
Korean students had never heard of Murphy’s Law. It appeared to be a new experience 
for them to plan, and then to revise the plans, when things did not go according to the 
original plan. 

The success of the course project depended upon frequent and quick 
communication among team members and with the instructor. E-mail and an electronic 
bulletin board were effective ways to maintain communication between the instructor 
and the students. The teams themselves did not use these two electronic 
communication strategies as planned by the instructor. When teams have most of their 
communication with electronic bulletin boards or e-mail that “copies” the instructor it is 
relatively easy for the instructor to monitor the teams’ progress. Korean students 
preferred to use cell phones for voice communication, leaving the instructor “out of the 
loop.” 

The teaching techniques that were learned in the psychometrics class that 
applied well in the advanced research methods class include the following. 

□ Topic Selection: Be sure the topic is of interest and relevant to the students. 

□ Teams of four to seven students should be composed of students with a mix of 
previous psychology coursework and abilities. 

□ Projects should be clearly divided into sequential parts. 

□ Teams need frequent and timely feedback on project activities. 

□ Students should be given an opportunity to describe how the project contributed 
to their understanding of the course content. 

Cross-Cultural Considerations 

Ethical review boards are not a part of the Korean University system and not 
used for the review of psychological research at Korea University. Furthermore, the use 
of informed consent with signed forms is inconsistent with the culture. For those 
reasons, I (as a visiting U.S. lecturer) decided to engage in only the types of research 
that do not require ethical review by University Review Boards under U.S. policy and 
practice and which also did not require review according to my home institution’s policy. 
The students were instructed in the ethics of survey research according to the American 
Psychological Association with the caveat that some principles may be culturally 
specific. The goal of treating research participants with dignity and respect was 
stressed. 

English was the language of instruction and the students were competent in both 
English listening and speaking. The research was appropriately conducted in the 
Korean language because of the population to be sampled, and more importantly 
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because of the issue under study. To attempt to study attitudes toward Korean 
reunification in a foreign language, and that language being English, would be likely to 
bias the results. The main problem with using the Korean language was for the 
instructor who needed to understand the project. The students wrote their original items 
in the Korean language and then made as close to verbatim translations for the 
professor as were possible. Some Korean concepts do not readily translate into 
English. In some cases the students simply did not know the English word. In many 
more there is no English word with the same meaning. In a few cases there may not be 
a similar Western concept. 

Korean collectivism made the team project particularly appropriate. Individual 
attention for performance, either positive or negative, is embarrassing and culturally 
inappropriate in the classroom context. Thus, individual active learning projects would 
be very difficult to introduce. Normally shy Koreans seem willing to do many unusual 
things when on teams for the benefit of the team’s performance. Evidence for this was 
the popularity of Korean TV game shows that feature teams of three or four people in 
competition doing many silly things for the camera and studio audience. None of the 
Korean students indicated discomfort with the team process. In contrast, in my 
American classes there are always some students who prefer to work alone. They 
especially prefer to have their grades based upon only their own work and complain that 
the team project will be an unfair way for them to be graded. However, it was 
unnecessary to explain to the Korean students that part of their future success depends 
upon their ability to work effectively in teams. The close bond between people who 
belong to the same group is one of those Korean words for which there is no exact 
English equivalent. Students in individualistic societies will probably need more explicit 
instruction in effective teamwork than did the Korean students. 
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